This PR implements InstantMesh for 3D meshing using multiview images. The torch-cuda version with textures is shown below.

Input is nx3xhxw multiview video and the output is a 3D mesh mesh.obj file with vertices, faces.
HW environment specs are as follows.

Mindspore Version:               2.3.1 release
CANN Version:                    7.3
Ascend Driver:                   23.0.rc3.6

3D Mesh Demos

Inference demos with the pretrained checkpoint can be found in the readme, with any HTML renderer such as VSCode it can be viewed and interacted with smoothly as shown below.

These links can be found here for easy access.

• akun
• anya

Input multiview images are illustrated here, respectively. Kindly notice that the input multiview images can be either retrieved viav the SV3D pipeline or Zero123++ pipeline as in the original implementation, the paper's core contribution is the process of 3D meshing out of multiview images.

Limitations in Inference

1. Cache Miss in mint.unique() and other ACLNN Operators

As introduced in the Readme, InstantMesh extracts isosurface for meshing using FlexiCubes, which essentially requires a unique operation to determine the surfaces of an object. Unfortunately, the operator mint.unique() remains not supported by ACLNN but only AICPU, and hence leads to the program being stuck by cache misses, as told by the MindSpore framework colleagues. Once the mint.unique() is fully supported by A+M, we will implement the 3D meshing with FlexiCubes for higher resolution.

(Update on August 1st: Turns out the CANN operator aclnnUniqueDim takes too long. Interacting with the framework and CANN team for solutions.)

Workaround: 3D Meshing with the Raw Triplane Features using Marching Cubes

For the reasons above, we have to find a workaround to extract 3D meshes from multiview images (from SV3D in our case). Here in this PR, since we already have a rough SDF extracted from the SDF MLP heads with the triplane features input, a straightforward way is to take the rough SDF and feed it to the classic isosurface extraction method, such as the Marching Cubes. As the optimization in the Marching Cubes lacks degrees of freedom to represent high-quality meshes, it tends to use more vertices (naturally more faces, triangles) to fit an irregular 3D shape, especially when the 3D shape cannot be approximated to a surface. Details can be found in the FlexiCubes' paper.

FlexiCubes	Marching Cubes

2. CUDA Extension for Rasterization

InstantMesh uses nvdiffrast for uv map rasterization and 2D rendering for the FlexiCube 3D volume. Bypass this for now.